Search for: All records

Accurate simulation of earthquake scenarios is essential for advancing seismic hazard analysis and risk mitigation strategies. At the San Diego Supercomputer Center (SDSC), our research focuses on optimizing the performance and reliability of large-scale earthquake simulations using the AWP-ODC software. By implementing GPU-aware MPI calls, we enable direct data processing within GPU memory, eliminating the need for explicit data transfers between CPU and GPU. This GPU-aware MPI achieves nearly ideal parallel efficiency at full scale across both Nvidia and AMD GPUs, leveraging the MVAPICH-PLUS support on Frontier at Oak Ridge National Laboratory and Vista at the Texas Advanced Computing Center. We utilized the MVAPICH-Plus 4.0 compiler to enable ZFP compression, which significantly enhances inter-node communication efficiency – a critical improvement given the communication bottleneck inherent in large-scale simulations. Our GPU-aware AWP-ODC versions include linear forward, topography and nonlinear Iwan-type solvers with discontinuous mesh support. On the Frontier system with MVAPICH 4.0, Hip-aware MPI calls on MI250X GPUs deliver nearly ideal weak-scaling speedup up to 8,192 nodes for both linear and topography versions. On TACC’s Vista system, CUDA-aware MPI calls on GH200 GPUs substantially outperform their non-GPU-aware counterparts across all three solver versions. This poster will present a detailed evaluation of GPU-aware AWP-ODC using MVAPICH, including the impact of ZFP message compression compared to the native versions. Our results highlight the importance of Mvapich support for GPU-ware MPI and on-the-fly compression techniques for accelerating and scaling earthquake simulations. 

We have implemented GPU-aware support across all AWP-ODC versions and enhanced message-passing collective communications for this memory-bound finite-difference solver. This provides cutting-edge communication support for production simulations on leadership-class computing facilities, including OLCF Frontier and TACC Vista. We achieved significant performance gains, reaching 37 sustained Petaflop/s and reducing time-to-solution by 17.2% using the GPU-aware feature on 8,192 Frontier nodes, or 65,336 MI250X GCDs. The AWP-ODC code has also been optimized for TACC Vista, an Arm-based NVIDIA GH200 Grace Hopper Superchip, demonstrating excellent application performance. This poster will showcase studies and GPU performance characteristics. We will discuss our verification of GPU-aware development and the use of high-performance MVAPICH libraries, including on-the-fly compression, on modern GPU clusters. 

Land surface temperature (LST) derived from satellite observations and weather modeling has been widely used for investigating Earth surface-atmosphere energy exchange and radiation budget. However, satellite-derived LST has a trade-off between spatial and temporal resolutions and missing observations caused by clouds, while there are limitations such as potential bias and expensive computation in model calibration and simulation for weather modeling. To mitigate those limitations, we proposed a WRFM framework to estimate LST at a spatial resolution of 1 km and temporal resolution of an hour by integrating the Weather Research and Forecasting (WRF) model and MODIS satellite data using the morphing technique. We tested the framework in eight counties, Iowa, USA, including urban and rural areas, to generate hourly LSTs from June 1st to August 31st, 2019, at a 1 km resolution. Upon evaluation with in-situ LST measurements, our WRFM framework has demonstrated its ability to capture hourly LSTs under both clear and cloudy conditions, with a root mean square error (RMSE) of 2.63 K and 3.75 K, respectively. Additionally, the assessment with satellite LST observations has shown that the WRFM framework can effectively reduce the bias magnitude in LST from the WRF simulation, resulting in a reduction of the average RMSE over the study area from 4.34 K (daytime) and 4.12 K (nighttime) to 2.89 K (daytime) and 2.75 K (nighttime), respectively, while still capturing the hourly patterns of LST. Overall, the WRFM is effective in integrating the complementary advantages of satellite observations and weather modeling and can generate LSTs with high spatiotemporal resolutions in areas with complex landscapes (e.g., urban).